Business entities continue to distribute geographically their operations. However, at the same time, many business entities desire to consolidate their computing infrastructure to a single geographic location in order to simplify maintenance and administration. These competing efforts often require that a remote business operation communicate with a centralized data center over a wide area network (WAN). Delays associated with communication over a WAN typically results in a user experience that is not satisfying to the user. Therefore techniques for accelerating the network traffic over the WAN are often employed.
One such acceleration technique is disk-based traffic caching, i.e., maintaining a history of previously-transmitted network traffic on disk in order to identify and send in place of the network traffic a tag or other identifier over the WAN. However, traditional disk-based caching techniques typically fail to provide acceptable performance because cache index entries, which are stored in memory, tend to scale with the number of cache entries, meaning that traditional disk-based caches consume large amounts of memory. In some cases, the size of the disk cache is artificially constrained because there is not enough memory to properly index the cache.
Another acceleration technique is traffic caching, i.e., maintaining a history of previously-transmitted network traffic in order to identify and send in place of the network traffic a tag or other identifier over the WAN. Such caches generally associate a “freshness” value with each cached entry. If a requested object is stored in the cache, and its freshness value indicates that the cached object is still valid, the object is sent to the client. However, this approach generally suffers from a drawback that the freshness value may not accurately reflect whether the object has changed. Therefore, reliance on freshness values can result in the cache transmitting outdated information to a user, perhaps many times, before the freshness value expires.
Another such acceleration technique is traffic caching, i.e., maintaining a history of previously-transmitted network traffic on disk in order to identify and send in place of the network traffic a tag or other identifier over the WAN. This approach, however, requires construction of a compression history in order to function well and such construction may require that many different files and objects are requested before providing a user with any perception of acceleration.
These caches generally associate a “freshness” value with each cached entry. If a requested object is stored in the cache, and its freshness value indicates that the cached object is still valid, the object is sent to the client. However, this approach generally suffers from a drawback that many web objects are delivered without an associated freshness value. In these cases a browser application or cache typically assigns a freshness value to the object and, in most cases, the freshness value is selected according to a rule that does not vary with the type of object encountered. This approach is not optimal since it can result both in the provision of stale objects as well as the validation or freshening of cached objects that are not stale.
If a requested object is stored in the cache, and its freshness value indicates that the cached object is still valid, the object is sent to the client. However, this approach generally suffers from a drawback that the freshness value may not accurately reflect whether the object has changed. Therefore, reliance on freshness values can result in the cache transmitting outdated information to a user, perhaps many times, before the freshness value expires.
In some cases, proxy servers providing cache memories may be deployed in an attempt to ameliorate the delays encountered by WAN traffic, i.e., a proxy server may be able to respond to a user request with data stored in a cache, rather than requiring the user to retrieve the requested file or object from the data center. However, proxy servers pose a security risk for businesses because they typically do not require user authentication. Therefore, a proxy server may incorrectly respond to a request from a user that does not have authority to retrieve a particular file or object, compromising data security.
One technique that may be used is prefetching, i.e., identifying a file or object that is likely to be requested by a user and requesting that file or object before it is actually requested. However, this technique usually does not increase performance as much as might be expected because the prefetching traffic competes with actual requests for files and objects. That contention slows down actual requests and can actually exacerbate the delays perceived by users.
In some cases, the “freshness” of an object is validated before it is transmitted to the client. For example, a conditional HTTP GET command may be issued to determine if a requested object is still valid. However, validating an object takes almost as long as fetching it and, therefore, encountering a series of stale objects results in poor performance because the validation happens while the user is waiting.
Each request sent by a user begins with resolving a Uniform Resource Locator (URL) to an IP address. In some cases this resolution may take more than a second to complete, which appears to the user as a delay. Acceleration of DNS resolution would result in a better user experience.